Image processing apparatus, image processing method, and program

ABSTRACT

Noise is sufficiently reduced in a moving image. 
     A determination section determines, each time an input image is input, whether a change amount of a pixel value distribution in the input image and an output image that is output is smaller than a predetermined threshold value. A mixing ratio supply section supplies a value that becomes larger as the number of times the change amount is sequentially determined to be smaller than the predetermined threshold value becomes larger, the value serving as a mixing ratio of the output image in mixing of the input image and the output image. A mixing section mixes, each time the input image is input, the input image and the output image based on the supplied mixing ratio and outputs the input image and the output image to serve as a new output image.

TECHNICAL FIELD

The present technology relates to an image processing apparatus and animage processing method, and a program causing a computer to execute themethod. Specifically, the present technology relates to an imageprocessing apparatus and an image processing method with which noise ofmoving images is reduced, and a program causing a computer to executethe method.

BACKGROUND ART

In order to reduce noise mixed in moving images, there has been usedfrom the past a technology called 3-dimensional noise reduction (3DNR).The 3DNR is a technology to reduce noise by mixing two two-dimensional(2D) frames that are continuous on a time axis. In the 3DNR, in general,each time an input frame is input, an image processing apparatus mixesan output frame previously output and the current input frame, togenerate a current output frame. In this mixing, the following imageprocessing apparatus is proposed: the previous output frame and thecurrent input frame are compared with each other to obtain a motionamount, and as the motion amount becomes larger, a mixing ratio of theprevious output frame is made lower (see, for example, Patent Document1).

Patent Document 1: Japanese Patent Application Laid-open No. 2010-4266

SUMMARY OF INVENTION Problem to be Solved by the Invention

In the related art described above, however, there is a possibility thatthe noise cannot be sufficiently reduced. Namely, since the imageprocessing apparatus described above determines the mixing ratioaccording to the motion amount, when a state where the motion amount isconstant is continued, each of the output frames is mixed according tothe constant mixing ratio. When the mixing ratio is constant over aplurality of input frames, there is a possibility that those inputframes are not mixed with the output frames at an equal ratio.

For example, when continuous input frames F1, F2, and F3 are input inchronological order, and the mixing ratio of output frames is a constantvalue α, the input frame F1 is first output to serve as an output frameF1′ without change. Next, the previous output frame F1′ (i.e., inputframe F1) having a ratio of α and the input frame F2 having a ratio of(1−α) are mixed, to output an output frame F2′. The previous outputframe F2′ having a ratio of α and the input frame F3 having a ratio of(1−α) are then mixed, to output an output frame F3′. Since the ratio ofthe input frames F1 and F2 in the output frame F2′ is α:(1−α), the ratioof the input frames F1, F2, and F3 in the output frame F3′ isα²:α(1−α):(1−α). For the mixing ratio α, a value smaller than “1” isgenerally set, and thus older input frames have smaller ratios in theoutput frame. Therefore, there is a possibility that when the mixingratio is constant, the noise cannot be sufficiently reduced.

Further, when intensive motions occur in a moving image, there is apossibility that a difference in ratio between the input frames in theoutput frame is increased. For example, it is conceived a case where achange between the input frames F1 and F2 is large, and the next inputframe F3 and frames subsequent thereto have small changes betweenadjacent frames. In the case, since the change between the input framesF1 and F2 is large, in the mixing of the input frames F1 and F2, theinput frame F1 is mixed at a high mixing ratio α. In contrast to this,the input frame F2 and the input frames subsequent thereto have smallchanges between adjacent frames and are mixed at a lower mixing ratio.As a result, the input frame F1 is mixed at a higher ratio than theinput frames subsequent to the input frame F1, and a noise reducingeffect is lost.

The present technology has been made in view of the circumstances asdescribed above, and it is an object of the present technology tosufficiently reduce noise in a moving image.

Means for Solving the Problem

The present technology has been made in order to eliminate the problemsdescribed above. According to a first aspect of the present technology,there is provided an image processing apparatus, an image processingmethod, and a program causing a computer to execute the image processingmethod. The image processing apparatus includes: a determination sectionthat determines, each time an input image is input, whether a changeamount of a pixel value distribution in the input image and an outputimage that is output is smaller than a predetermined threshold value; amixing ratio supply section that supplies a value that becomes larger asthe number of times the change amount is sequentially determined to besmaller than the predetermined threshold value becomes larger, the valueserving as a mixing ratio of the output image in mixing of the inputimage and the output image; and a mixing section that mixes, each timethe input image is input, the input image and the output image based onthe supplied mixing ratio and outputs the input image and the outputimage to serve as a new output image. Thus, the following action iscaused: the input image and the output image are mixed at a mixing ratiothat becomes larger as the number of times the change amount issequentially determined to be smaller than the predetermined thresholdvalue becomes larger.

Further, in the first aspect, the mixing ratio supply section mayinclude an upper limit value generation section that generates an upperlimit value that becomes larger as the number of times becomes larger,an acquisition section that acquires a value that becomes smaller as thechange amount becomes larger, and a limitation section that supplies,when the acquired value is larger than the upper limit value, the upperlimit value to serve as the mixing ratio, and supplies, when theacquired value does not exceed the upper limit value, the acquired valueto serve as the mixing ratio. Thus, the following action is caused: avalue limited by the upper limit value that becomes larger as the numberof times becomes larger is supplied to serve as the mixing ratio.

Further, in the first aspect, the mixing ratio supply section may supplythe mixing ratio that becomes larger as the number of times becomeslarger and at which a proportion of each of the input images mixed inthe output image is made uniform. Thus, the following action is caused:the mixing ratio that becomes larger as the number of times becomeslarger and at which a proportion of each of input images mixed in theoutput image is made uniform is supplied.

Further, in the first aspect, each of the input image and the outputimage may be an image including a plurality of areas each including aplurality of pixels, the determination section may determine, per area,whether the change amount is smaller than the predetermined thresholdvalue, and the mixing ratio supply section may acquire, per area, thenumber of times the change amount is sequentially determined to besmaller than the predetermined threshold value, and supply the mixingratio that becomes larger as a minimum value of the numbers of timesacquired per area becomes larger. Thus, the following action is caused:a larger mixing ratio is supplied as a minimum value of the numbers oftimes per area the change amount is sequentially determined to besmaller than the predetermined threshold value becomes larger.

Further, in the first aspect, each of the input image and the outputimage may be an image including a plurality of pixels, the determinationsection may determine, per pixel, whether the change amount is smallerthan the predetermined threshold value, and the mixing ratio supplysection may acquire, per pixel, the number of times the change amount issequentially determined to be smaller than the predetermined thresholdvalue, and supply the mixing ratio that becomes larger as a minimumvalue of the numbers of times acquired per pixel becomes larger. Thus,the following action is caused: a larger mixing ratio is supplied as aminimum value of the numbers of times per pixel the change amount issequentially determined to be smaller than the predetermined thresholdvalue becomes larger.

Further, in the first aspect, the image processing apparatus may furtherinclude: a replacement section that replaces, when the pixel includesinvalid data together with valid data indicating a pixel value, theinvalid data not applying to the valid data, the invalid data within thepixel per pixel based on the number of times; and an image holdingsection that holds the output image in which the invalid data isreplaced, in which the mixing ratio supply section may acquire thenumber of times from the held output image. Thus, the following actionis caused: the number of times is acquired from the output image inwhich the invalid data is replaced.

Further, according to a second aspect of the present technology, thereis provided an imaging apparatus including: a determination section thatdetermines, each time an input image is input, whether a change amountof a pixel value distribution in the input image and an output imagethat is output is smaller than a predetermined threshold value; a mixingratio supply section that supplies a value that becomes larger as thenumber of times the change amount is sequentially determined to besmaller than the predetermined threshold value becomes larger, the valueserving as a mixing ratio of the output image in mixing of the inputimage and the output image; a mixing section that mixes, each time theinput image is input, the input image and the output image based on thesupplied mixing ratio and outputs the input image and the output imageto serve as a new output image; and an image holding section that holdsthe output image that is output. Thus, the following action is caused:the input image and the output image are mixed at a mixing ratio thatbecomes larger as the number of times the change amount is sequentiallydetermined to be smaller than the predetermined threshold value becomeslarger.

Effect of the Invention

According to the present technology, it is possible to produce anexcellent effect in which noise can be sufficiently reduced in a movingimage.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a block diagram showing a configuration example of an imagingapparatus in a first embodiment.

FIG. 2 is a block diagram showing a configuration example of an imageprocessing section in the first embodiment.

FIG. 3 is a block diagram showing a configuration example of a mixingratio supply section in the first embodiment.

FIG. 4 is a graph showing an example of a mixing ratio corresponding toa motion amount in the first embodiment.

FIG. 5 is a graph showing an example of a limit value corresponding to acount value in the first embodiment.

FIG. 6 is a block diagram showing a configuration example of a mixingsection in the first embodiment.

FIG. 7 is a flowchart showing an example of image processing executed bythe image processing section 200 in the first embodiment.

FIG. 8 is a diagram showing an example of a moving image to be processedin the first embodiment.

FIG. 9 is a diagram showing an example of the count value, the limitvalue, and the mixing ratio in the first embodiment.

FIG. 10 is a graph showing an example of an IIR coefficient in the firstembodiment.

FIG. 11 is a block diagram showing a configuration example of an imageprocessing section in a second embodiment.

FIG. 12 is a diagram showing a data configuration example of pixel databefore the embedment of the count value in the second embodiment.

FIG. 13 is a diagram showing a data configuration example of pixel dataafter the embedment of the count value in the second embodiment.

FIG. 14 is a block diagram showing a configuration example of a motiondetermination section in the second embodiment.

FIG. 15 is a block diagram showing a configuration example of astationary frame number count section in the second embodiment.

FIG. 16 is a block diagram showing a configuration example of alimitation section in the second embodiment.

FIG. 17 is a block diagram showing a configuration example of a motiondetermination section in a third embodiment.

FIG. 18 is a block diagram showing a configuration example of astationary frame number count section in the third embodiment.

FIG. 19 is a block diagram showing a configuration example of a motiondetermination section in a fourth embodiment.

FIG. 20 is a block diagram showing a configuration example of astationary frame number count section in the fourth embodiment.

MODE(S) FOR CARRYING OUT THE INVENTION

Hereinafter, forms for carrying out the present technology (hereinafter,called embodiments) will be described. It should be noted that thedescription is provided in the following order.

1. First Embodiment (Example of mixing at larger mixing ratio in largercount value)2. Second Embodiment (Example of mixing at larger mixing ratio asminimum value of count values per pixel and of the entire image islarger)3. Third Embodiment (Example of mixing at larger mixing ratio as minimumvalue of count values per pixel and per area is larger)4. Fourth Embodiment (Example of mixing at larger mixing ratio asminimum value of count values of the entire image, per pixel, and perarea is larger)

1. First Embodiment Configuration Example of Imaging Apparatus

FIG. 1 is a block diagram showing a configuration example of an imagingapparatus 100 in a first embodiment of the present technology. Theimaging apparatus 100 is an apparatus that captures images, and includesan imaging lens 110, an imaging device 120, an analog front end 130, anA/D conversion section 140, a programmable gain amplifier 150, and acamera control section 160. Further, the imaging apparatus 100 includesan image processing section 200, a D/A conversion section 170, and adisplay section 180.

The imaging lens 110 is a lens for forming an image of an imaging targeton the imaging device 120. The imaging lens 110 includes lenses such asa focus lens and a zoom lens.

The imaging device 120 photoelectrically converts light from the imaginglens 110 and supplies a converted electric signal, which serves as animage signal, to the analog front end 130 via a signal line 129. Theimaging device 120 can be achieved by a CCD (Charge Coupled Device) orCMOS (Complementary Metal Oxide Semiconductor) sensor or the like.

The analog front end 130 performs processing, such as noise removingprocessing and amplification processing, on the analog image signal andsupplies the resultant signal to the A/D conversion section 140 via asignal line 139. The analog front end 130 includes, for example, acorrelated double sampling (CDS) circuit, and removes noise by that CDScircuit.

The A/D conversion section 140 converts the analog image signal intodigital image data and supplies the digital image data to theprogrammable gain amplifier 150 via a signal line 149. The image dataincludes pieces of pixel data, each of which indicates a pixel value ofeach pixel. It should be noted that the A/D conversion section 140 isconfigured to be provided on the outside of the analog front end 130,but the A/D conversion section 140 may be provided inside the analogfront end 130.

The programmable gain amplifier 150 amplifies the pixel value of eachpixel in the image data according to the control of the camera controlsection 160. The programmable gain amplifier 150 supplies the amplifiedimage data to the image processing section 200 via a signal line 159.

The image processing section 200 performs 3DNR on a moving image formedof the pieces of image data. The pieces of image data are input, asinput frames, to the image processing section 200 in chronologicalorder. Each time the input frame is input, the image processing section200 outputs image data, which is obtained by mixing the current inputframe and a previous output frame to serve as a current output frame, tothe D/A conversion section 170 via a signal line 209. When the firstinput frame is input, a previous output frame is not present, and thusthe input frame is output as an output frame without change. The imageprocessing section 200 mixes two frames that are continuous inchronological order, to thus remove noise in the moving image.

It should be noted that the image processing section 200 can furtherperform image processing such as demosaic processing, white balanceprocessing, and gamma correction processing, in addition to 3DNR. Theorder in which those processing are performed is arbitrarily set.

The camera control section 160 controls the entire imaging apparatus100. According to an operation of a user (a press of a start buttonetc.), the camera control section 160 controls the imaging device 120and the analog front end 130 to start generating a plurality of imagesignals. Those image signals are generated in preset constant cycles(for example, 1/60 seconds). According to an operation of the user (apress of an end button etc.), the camera control section 160 terminatesthe generation of the image signals. Further, the camera control section160 controls an amplification factor of the pixel values in theprogrammable gain amplifier 150 to be a preset value.

The D/A conversion section 170 converts the digital image data intoanalog image signals and supplies the analog image signals to thedisplay section 180 via a signal line 179. The display section 180displays an image based on the image signals.

It should be noted that the imaging apparatus 100 can further include arecording section that records the image data in a recording medium suchas a memory.

[Configuration Example of Image Processing Unit]

FIG. 2 is a block diagram showing a configuration example of the imageprocessing section 200 in the first embodiment. The image processingsection 200 includes a motion determination section 210, a stationaryframe number count section 220, a mixing ratio supply section 230, amixing section 250, and a frame buffer 270.

Each time the input frame is input, the motion determination section 210compares a previous output frame and the current input frame anddetermines the presence or absence of a motion. In the determination onthe presence or absence of a motion, the motion determination section210 first reads the previous output frame from the frame buffer 270 andobtains as a motion amount dS a change amount of a pixel valuedistribution of that output frame and the current input frame. Forexample, the motion determination section 210 obtains differences inpixel values between pixels having the same coordinates in the outputframe and the input frame. The motion determination section 210 thencalculates statistics of those differences (for example, mean value) toserve as a motion amount dS.

After obtaining the motion amount dS, the motion determination section210 determines that there is no motion when the motion amount dS issmaller than a predetermined threshold value Th_f, and determines thatthere is a motion when the motion amount dS is equal to or larger thanthe threshold value Th_f. The motion determination section 210 suppliesthe motion amount dS to the mixing ratio supply section 230 and suppliesa determination result on the presence or absence of a motion to thestationary frame number count section 220. In such a manner, an approachto obtain a motion from the differences in pixel values between theframes is called an inter-frame difference method.

It should be noted that the motion determination section 210 is anexample of a determination section described in the Claims. Further, themotion determination section 210 can determines the presence or absenceof a motion by using an approach other than the inter-frame differencemethod. For example, the motion determination section 210 may determinethe presence or absence of a motion by using a block matching algorithm.

When the block matching algorithm is used, each of the input frame andthe output frame is handled as an image formed of a plurality of blockseach having a predetermined shape (for example, square of 8×8 pixels).The motion determination section 210 obtains two blocks having a highcorrelation in the input frame and the output frame, generates a vectorfrom one of the blocks to the other block to serve as a motion vector,and calculates the magnitude of the vector to serve as a motion amountdS. In the block matching algorithm, a motion amount is calculated foreach block within the frame, and thus those statistics (for example,mean value) are output to serve as the motion amount dS of the entirescreen.

The stationary frame number count section 220 counts the number of timesno motion is sequentially determined, and generates a counted value toserve as a count value CNTf. The stationary frame number count section220 holds the count value CNTf. The stationary frame number countsection 220 increments the count value CNTf when it is determined thatthere is no motion, and sets the count value CNTf to be an initial value(for example, “0”) when it is determined that there is a motion. Thestationary frame number count section 220 supplies the count value CNTfto the mixing ratio supply section 230.

It should be noted that the stationary frame number count section 220increments the count value CNTf, but an incremental value is not limitedto “1”. Further, the stationary frame number count section 220increments the count value CNTf when there is no motion, but thestationary frame number count section 220 may be a down counter thatdecrements the count value CNTf when there is no motion.

The mixing ratio supply section 230 supplies a mixing ratio α′ in whichthe proportion of an output image is made higher as the number of timesno motion is sequentially determined (i.e., count value CNTf) becomeslarger. For example, the mixing ratio supply section 230 generates alarger mixing ratio α as the motion amount dS becomes smaller, andgenerates a lager upper limit value L as the count value CNTf becomeslarger. The mixing ratio supply section 230 then limits the mixing ratioα so as not to exceed the upper limit value L and sets a limited valueto be the mixing ratio α′. Here, the mixing ratios α and α′ are each avalue indicating the proportion of the output frame in the mixing of theinput frame and the output frame, and for example, real numbers of 0 to1 is set therefor. The mixing ratio supply section 230 supplies the setmixing ratio α′ to the mixing section 250.

The mixing section 250 mixes the previous output frame and the currentinput frame based on the mixing ratio α′. Each time the input frame isinput, the mixing section 250 reads the previous output frame from theframe buffer 270 and mixes that output frame and the current input framebased on the mixing ratio α′. Specifically, the mixing section 250 mixesthe frames by using the following expression. It should be noted thatwhen the first input frame is input, a previous output frame is notpresent, and thus the mixing ratio α′ is set to “0”.

Y _(t)(x,y)=α_(t) ′×Y _(t−1)(x,y)+(1−α_(t)′)×X _(t)(x,y)  Expression 1

In the above expression, Y_(t)(x,y) and Y_(t−1)(x,y) are pixel values ofpixels having coordinates (x,y) in the t-th and (t−1)-th output frames.x is a horizontal coordinate in the frame, and y is a verticalcoordinate. Here, the number of pixels in the horizontal direction ofthe frame is M (M is an integer), and x is a value of 0 to M−1. Further,the number of pixels in the vertical direction of the frame is N (N isan integer), and y is a value of 0 to N−1. X_(t)(x,y) is a pixel valueof a pixel having coordinates (x,y) in the t-th input frame. α_(t)′ is amixing ratio that is set t-th.

In the filter expressed by the expression 1, an output signal (i.e.,output frame) is fed back, and thus a value that is not zero is outputwith respect to an impulse over an unlimited period of time. Therefore,such a filter is called an infinite impulse response (IIR) filter.

The mixing section 250 outputs the generated current output frame to theD/A conversion section 170 and also causes the frame buffer 270 to holdthe generated current output frame. The frame buffer 270 holds theoutput frame. It should be noted that the frame buffer 270 is an exampleof an image holding section described in the Claims.

[Configuration Example of Mixing Ratio Supply Unit]

FIG. 3 is a block diagram showing a configuration example of the mixingratio supply section 230 in the first embodiment. The mixing ratiosupply section 230 includes a mixing ratio generation section 231, anupper limit value generation section 242, and a mixing ratio limitationsection 243.

The mixing ratio generation section 231 generates a larger mixing ratioα as the motion amount dS becomes smaller. The mixing ratio generationsection 231 generates the mixing ratio α by using a function expressedby the following expression, for example.

[Math.  1] $\begin{matrix}{\alpha = \left\{ \begin{matrix}\alpha_{\max} & \left( {{dS} < {Th\_ s}} \right) \\{{{- a} \times {dS}} + b} & \left( {{dS} \geqq {Th\_ s}} \right)\end{matrix} \right.} & {{Expression}\mspace{14mu} 2}\end{matrix}$

In the above expression, a and b are each a positive real number.α_(max) is the upper limit value of the mixing ratio α, and a realnumber that is larger than “0” and smaller than “1” is set therefor.Th_s is a threshold value of the motion amount dS. The Th_s is set to,for example, the same value as the threshold value Th_f used for thedetermination on the presence or absence of a motion. It should be notedthat those threshold values may be set to different values.

It should be noted that the mixing ratio generation section 231 mayobtain the mixing ratio α by an expression other than Expression 2 aslong as a larger mixing ratio α can be generated as the motion amount dSbecomes smaller. Further, the mixing ratio generation section 231obtains the mixing ratio α by a computation, but the mixing ratiogeneration section 231 is not limited to this configuration. Forexample, the mixing ratio generation section 231 may be configured toinclude a table in which the mixing ratio α obtained in advance fromExpression 2 or the like and the motion amount dS are associated witheach other, and read the mixing ratio α corresponding to the motionamount dS from that table. Alternatively, if the motion amount dS isequal to or larger than the threshold value Th_s, the mixing ratiogeneration section 231 may return the value of “0”, and if not, themixing ratio generation section 231 may use a function for returning theα_(max), to generate a mixing ratio α without performing a computation.It should be noted that the mixing ratio generation section 231 is anexample of an acquisition section described in the Claims.

The upper limit value generation section 242 generates the upper limitvalue L from the count value CNTf. The upper limit value generationsection 242 calculates the upper limit value L by using the followingexpression, for example, and supplies the upper limit value L to themixing ratio limitation section 243.

L=(CNTf+1)/(CNTf+2)  Expression 3

It should be noted that the upper limit value generation section 242 mayobtain the upper limit value L by an expression other than Expression 3as long as a smaller upper limit value L can be generated as the countvalue CNTf becomes larger. Further, the upper limit value generationsection 242 obtains the upper limit value L by a computation, but theupper limit value generation section 242 is not limited to thisconfiguration. For example, the upper limit value generation section 242may be configured to include a table in which the upper limit value Lobtained in advance from Expression 3 or the like and the count valueCNTf are associated with each other, and read the upper limit value Lcorresponding to the count value CNTf from that table.

The mixing ratio limitation section 243 limits the mixing ratio α so asnot to exceed the upper limit value L. When the mixing ratio α is equalto or larger than the upper limit value L, the mixing ratio limitationsection 243 outputs to the mixing section 250 the upper limit value L toserve as a limited mixing ratio α′. On the other hand, when the mixingratio α is smaller than the upper limit value L, the mixing ratiolimitation section 243 outputs to the mixing section 250 the mixingratio α, without change, to serve as the mixing ratio α′.

It should be noted that the mixing ratio supply section 230 supplies themixing ratio α′ of the output frame, but the mixing ratio supply section230 may supply a mixing ratio (1−α′) of the input frame instead of α′.In the case, the mixing ratio generation section 231 generates a lagermixing ratio (1−α) as the motion amount dS becomes larger. Further, theupper limit value generation section 242 generates a lower limit valuethat becomes larger as the count value CNTf becomes larger. The mixingratio limitation section 243 then limits the mixing ratio (1−α) so asnot to be smaller than the lower limit value.

FIG. 4 is a graph showing an example of the mixing ratio α correspondingto the motion amount dS in the first embodiment. As exemplified in thefigure, a smaller mixing ratio α is generated as the motion amount dSbecomes larger. Further, when the motion amount dS is smaller than thethreshold value Th_s, the mixing ratio α_(max) of the upper limit isgenerated.

FIG. 5 is a graph showing an example of the upper limit value Lcorresponding to the count value CNTf in the first embodiment. Asexemplified in the figure, a lager upper limit value L is generated asthe count value CNTf becomes larger.

[Configuration Example of Mixing Unit]

FIG. 6 is a block diagram showing a configuration example of the mixingsection 250 in the first embodiment. The mixing section 250 includes asubtractor 251, multipliers 252 and 253, and an adder 254.

The subtractor 251 subtracts one of two input values from the otherinput value. A value of “1” and the mixing ratio α′ are input to thesubtractor 251. The subtractor 251 subtracts the mixing ratio α′ from“1” and supplies a subtraction result (1−α′) to the multiplier 252.

Each of the multipliers 252 and 253 multiplies two input values. The(1−α′) and a pixel value of a pixel within the input frame are input tothe multiplier 252. One input frame includes a plurality of (forexample, M×N) pixel values, and those pixel values are sequentiallyinput in a predetermined order. Each time the pixel value is input, themultiplier 252 multiplies that pixel value and (1−α′) together andsupplies a multiplication result to the adder 254.

The mixing ratio α′ and a pixel value of a pixel within the output frameread from the frame buffer 270 are input to the multiplier 253. Eachtime a pixel value is input, the multiplier 253 multiplies that pixelvalue and the mixing ratio α′ together and supplies a multiplicationresult to the adder 254.

The adder 254 adds two input values. The adder 254 adds themultiplication result of the multiplier 252 and the multiplicationresult of the multiplier 253 and supplies to the frame buffer 270 anaddition result to serve as a pixel value of the output frame.

[Example of Operation of Imaging Apparatus]

FIG. 7 is a flowchart showing an example of image processing executed bythe image processing section 200 in the first embodiment. The imageprocessing is executed each time an input frame within a moving image isinput to the image processing section 200, for example. The imageprocessing section 200 detects the motion amount dS from a current inputframe and a previous output frame and determines the presence or absenceof a motion (Step S901).

The image processing section 200 counts the number of times no motion issequentially determined and generates the count value CNTf (Step S902).The image processing section 200 generates the mixing ratio α from themotion amount dS (Step S903). The image processing section 200 thengenerates the upper limit value L from the count value CNTf and limitsthe mixing ratio α by that upper limit value L (Step S904).

The image processing section 200 mixes the input frame and the outputframe based on the limited mixing ratio α′ for output (Step S905). AfterStep S905, the image processing section 200 terminates the imageprocessing.

FIG. 8 is a diagram showing an example of a moving image to be processedin the first embodiment. As exemplified in the figure, the moving imageincludes 64 input frames having frame numbers “0” to “63”. It is assumedthat the input frame having the frame number “0” is generated with theflash on and the luminance of that frame is high as a whole. Further, itis assumed that the input frames having the frame numbers “0” to “63”each have a lower luminance than the frame number “0”, and a changeamount of a luminance distribution between adjacent frames is small.

FIG. 9 is a diagram showing an example of the count value, the upperlimit value, and the mixing ratio in the first embodiment. Here, it isassumed that the moving image of FIG. 8 is input. Since a previousoutput frame is not present at the time the input frame having the framenumber “0” is input, the mixing ratio α′ is set to “0”.

Next, at the time the input frame having the frame number “1” is input,the motion amount dS of the current input frame with respect to theprevious output frame is large. Thus, the mixing ratio α is set to “0”.The same holds true for the limited mixing ratio α′.

At the time the input frame having the frame number “2” is input, themotion amount dS of the current input frame with respect to the previousoutput frame is small. Thus, the mixing ratio α is set to a relativelylarge “0.97”. The number of times no motion is sequentially determinedis once, and thus the upper limit value L is set to approximately “0.67”by Expression 3. Therefore, the limited mixing ratio α′ is “0.67”.

Since the motion amount dS is small also after the frame number “3”, arelatively large mixing ratio α is set. It should be noted that theupper limit value L becomes larger as the number of times no motion issequentially determined (i.e., count value CNTf) is increased, based onExpression 3, and the limited mixing ratio α′ also becomes larger.

In such a manner, a larger mixing ratio α′ is set as the number of timesno motion is sequentially determined becomes larger.

FIG. 10 is a graph showing an example of an IIR coefficient in the firstembodiment. In the figure, the vertical axis represents an IIRcoefficient, and the horizontal axis represents a frame number. Here,the IIR coefficient is a coefficient indicating a proportion of eachinput frame in the output frame. The IIR coefficient is expressed by thefollowing expressions 4 to 7, for example.

$\begin{matrix}{\left\lbrack {{Math}.\mspace{14mu} 2} \right\rbrack {Y_{n} = {{{\alpha_{1} \times \alpha_{2} \times \cdots \mspace{14mu} \alpha_{n} \times X_{0}} + {\left( {1 - \alpha_{1}} \right) \times \alpha_{2} \times \cdots \mspace{14mu} \alpha_{n} \times X_{1}} + \cdots + {\left( {1 - \alpha_{n - 1}} \right) \times \alpha_{n} \times X_{n - 1}} + {\left( {1 - \alpha_{n}} \right) \times X_{n}}} = {{a_{0}X_{0}} + {b_{1}X_{2}} + \cdots + {b_{n - 1} \times X_{n - 1}} + {c_{n}X_{n}}}}}} & {{Expression}\mspace{14mu} 4} \\{\left\lbrack {{Math}.\mspace{14mu} 3} \right\rbrack {a_{0} = {\alpha_{1} \times \alpha_{2} \times \cdots \times \alpha_{n}}}} & {{Expression}\mspace{14mu} 5} \\{\left\lbrack {{Math}.\mspace{14mu} 4} \right\rbrack {b_{k} = {\left( {1 - \alpha_{k}} \right){\prod\limits_{l = {k + 1}}^{n}\; \alpha_{l}}}}} & {{Expression}\mspace{14mu} 6} \\{\left\lbrack {{Math}.\mspace{14mu} 5} \right\rbrack {c_{n} = \left( {1 - \alpha_{n}} \right)}} & {{Expression}\mspace{14mu} 7}\end{matrix}$

In Expression 4, Y_(n) is a pixel value of a pixel in the n-th outputframe. α₀ to α_(n) are mixing ratios that are set 0th to n-th. X₀ toX_(n) are pixel values of pixels of the 0th to n-th input frames. α₀ isan IIR coefficient indicating a proportion of the 0th input frame. b_(k)(k is an integer of 1 to n−1) is an IIR coefficient indicating aproportion of the k-th input frame. c_(n) is an IIR coefficientindicating a proportion of the n-th input frame. In Expression 6, Π_(αl)(l is an integer of k+1 to n) indicates an infinite product of α_(l). Itshould be noted that the description of the coordinates of the pixels isomitted in Expressions 4 to 7. The description of the coordinates of thepixels is similarly omitted also in expressions after Expression 7.

A method of deriving Expression 4 will be described. When mixing isperformed at the mixing ratio α, a pixel value Y₁ of the 1st outputframe is expressed by the following expression based on Expression 1.

$\begin{matrix}\begin{matrix}{Y_{1} = {{\alpha_{1} \times Y_{0}} + {\left( {1 - \alpha_{1}} \right) \times X_{1}}}} \\{= {{\alpha_{1} \times X_{0}} + {\left( {1 - \alpha_{1}} \right) \times X_{1}}}}\end{matrix} & {{Expression}\mspace{14mu} 8}\end{matrix}$

A pixel value Y₂ of the 2nd output frame is expressed by the followingexpression based on Expression 8 above and Expression 1.

$\begin{matrix}\begin{matrix}{Y_{1} = {{\alpha_{2} \times Y_{1}} + {\left( {1 - \alpha_{2}} \right) \times X_{2}}}} \\{= {{\alpha_{2} \times \left( {{\alpha_{1} \times X_{0}} + {\left( {1 - \alpha_{1}} \right) \times X_{1}}} \right)} + {\left( {1 - \alpha_{2}} \right) \times X_{2}}}} \\{= {{\alpha_{1} \times \alpha_{2} \times X_{0}} + {\left( {1 - \alpha_{1}} \right) \times \alpha_{2} \times X_{1}} + {\left( {1 - \alpha_{2}} \right) \times X_{2}}}}\end{matrix} & {{Expression}\mspace{14mu} 9}\end{matrix}$

As exemplified in Expressions 8 and 9, the pixel value of the outputframe in each output frame subsequent to the 1st output frame can beexpressed by a sum of weighted values of the pixel values of theplurality of input frames. Similarly in Expressions 8 and 9, in the n-thoutput frame, when all the output frames are replaced with the inputframes by a convolution operation in which the function exemplified byExpression 1 is sequentially convoluted with respect to each pixel valueof the output frames before the (n−1)-th output frame, Expression 4 isderived.

Here, as exemplified in FIG. 8, a case is conceived in which, in the 64frames, there is a motion between the 0th and the 1st input frames andthere is no motion in the 2nd and subsequent frames. In this case, forexample, the mixing ratio α₀ of “0” is generated, and the mixing ratiosα₁ to α₆₃ of “31/32” (≈0.97) are generated. When those values aresubstituted into Expression 5, an IIR coefficient a₀ of “0” is acquired.

Further, an IIR coefficient b₁ is (31/32)⁶² (≈0.139) from Expression 6.An IIR coefficient b₂ is (1-31/32)×(31/32)⁶¹ (≈0.005). After that, basedon Expressions 6 and 7, a larger IIR coefficient is acquired as theframe number is increased. As a result, a trajectory of an alternatelong and short dash line in FIG. 10 is acquired. When the mixing ratio αis not limited, the trajectory indicates that the IIR coefficient, thatis, the proportion of the input frame is not made uniform. Further, whenthere is a motion in the 1st input frame, the IIR coefficient b₁ at thattime becomes extremely larger than the other IIR coefficients (b₂ etc.).Therefore, when mixing is performed at the mixing ratio α before thelimiting, a noise reducing effect is lost.

In contrast to this, when the mixing ratio α is limited by the upperlimit value L, a uniform IIR coefficient is acquired. For example, IIRcoefficients b_(k) and b_(k+1) are expressed by the followingExpressions 10 and 11 based on Expression 6.

b _(k)=(1−α_(k))×α_(k+1)×α_(k+2)× . . . ×α_(n)  Expression 10

b _(k+1)=(1−α_(k+1))×α_(k+2)× . . . ×α_(n)  Expression 11

From those Expressions 10 and 11, a relationship indicated by thefollowing expression is acquired.

b _(k)={(1−α_(k))×α_(k+1)/(1−α_(k+1))}×b _(k+1)  Expression 12

Here, it is assumed that the mixing ratios α_(k) and α_(k+1) limited bythe upper limit value L are equal to the upper limit value L. In thiscase, assuming from Expression 3 that the limited α_(k) is(CNTf+1)/(CNTf+2), the limited α_(k+1) is (CNTf+2)/(CNTf+3). When thoseare substituted into Expression 12, a value of(1−α_(k))×α_(k)/(1−α_(k+1)) is “1” in Expression 12. Therefore,b_(k)=b_(k+1) is established. Consequently, when mixing is performed bythe mixing ratio α limited by the upper limit value L, the IIRcoefficient is made uniform.

In the actual practice in FIG. 10, the mixing is performed under anideal condition in which the IIR coefficient, that is, the proportion ofthe input frame is uniform, as shown in FIG. 10 showing the IIRcoefficient of the case where the mixing ratio α is limited. Thus, noisewithin the moving image is sufficiently removed.

Here, it is presumed that a stationary state where there is no motioncontinues over the 64 frames, and a variance σ² of noise of each of the0th to 63rd input frames is all identical as shown in the followingexpression.

[Math. 6]

σ² =V(A ₀)=V(A ₁)=V(A ₂)= . . . =V(A ₆₃)  Expression 13

In the above expression, A₀ to A₆₃ represent all pixels of the 0th to63rd frames. Further, V( ) is a function that returns the variance ofnoise of the input frame.

Further, a mean of the pixel values of all pixels of all the 0th to 63rdinput frames is obtained from the following expression.

[Math. 7]

A =(A ₀ +A ₁ +A ₂ + . . . +A ₆₃)/64  Expression 14

A variance value of noise when the mixing ratio α is limited isexpressed by the following expression based on Expressions 13 and 14.

$\begin{matrix}{\left\lbrack {{Math}.\mspace{14mu} 8} \right\rbrack \begin{matrix}{{V\left( \overset{\_}{A} \right)} = {V\left( {\left( {A_{0} + A_{1} + A_{2} + \cdots + A_{63}} \right)/64} \right)}} \\{= {\left( {{V\left( A_{0} \right)} + {V\left( A_{1} \right)} + {V\left( A_{2} \right)} + \cdots \; + {V\left( A_{63} \right)}} \right)/64}} \\{= {\sigma^{2}/64}}\end{matrix}} & {{Expression}\mspace{14mu} 15}\end{matrix}$

As shown in the above expression, the variance of noise is reduced to1/64. Consequently, when the mixing ratio α is limited, since theproportion of the input frame is uniform, an improvement effect in anS/N ratio (Signal to Noise ratio) of approximately 18 decibels isacquired at the time 64 (=2⁶) frames have passed. Similarly, at the time2²¹ frames (i is an integer) have passed, an improvement effect in theS/N ratio of 6×i decibels is acquired.

In contrast to this, when the mixing ratio α is not limited, since theproportion of the input frame is not made uniform, the noise reducingeffect is made lower than the case where the mixing ratio α is limited.For example, at the time 64 frames have passed, an improvement effect inthe S/N ratio of only 14.7 dB is expected.

Further, when the mixing ratio α is not limited, it is found that the1st IIR coefficient b₁ is extraordinarily large, and the number ofconvergence frames until the effect imparted by the pixel value of the1st input frame on the output frame is lost is increased. Conversely,when the mixing ratio α is limited, as exemplified in FIG. 10, an idealIIR coefficient is applied, and the IIR converges by at most 64 frames.

As described above, according to the first embodiment of the presenttechnology, as the number of times no motion is sequentially determinedbecomes larger, the imaging apparatus 100 increases the proportion ofthe output image to mix the input image and the output image, and thusthe proportion of each input image in the output image can be madeuniform. Thus, the noise within the moving image can be sufficientlyreduced.

2. Second Embodiment Configuration Example of Imaging Apparatus

In the first embodiment, the image processing section 200 detects thepresence or absence of a motion of the entire screen, but a smaller unitof motion detection is desirable. An image processing section 200 of asecond embodiment is different from the first embodiment in that theimage processing section 200 detects both of a motion of the entirescreen and a motion per pixel.

FIG. 11 is a block diagram showing a configuration example of the imageprocessing section 200 in the second embodiment. The image processingsection 200 of the second embodiment is different from the firstembodiment in that the image processing section 200 further includes acounted-value embedment processing section 260 and a counted-valueextraction section 280.

Further, the motion determination section 210 of the second embodimentis different from the first embodiment in that the presence or absenceof a motion per pixel is further detected. The stationary frame numbercount section 220 of the second embodiment is different from the firstembodiment in that the stationary frame number count section 220 furthercounts, per pixel, the number of times no motion is sequentiallydetermined. Count values CNTp1 to CNTpm of respective pixels aresupplied to the mixing ratio supply section 230 and the counted-valueembedment processing section 260. Here, m is an integer and indicatesthe total number of pixels within the frame.

The mixing ratio supply section 230 of the second embodiment isdifferent from the first embodiment in that the mixing ratio supplysection 230 generates the upper limit value L from the count value CNTfof the entire screen and the count values CNTp1 to CNTpm of therespective pixels.

The counted-value embedment processing section 260 embeds the countvalue per pixel into pixel data in the output frame from the mixingsection 250. Specifically, the counted-value embedment processingsection 260 receives an invalid bit number from the camera controlsection 160. The invalid bit number indicates, in the pixel data, thenumber of bits that do not apply to valid bits indicating pixel values.Specifically, a value corresponding to a gain in the programmable gainamplifier 150 is set. For example, when the gain is 2^(x) times orlarger and is smaller than 2^(x+1), x bits become invalid bits.

The counted-value embedment processing section 260 replaces an invalidbit within the pixel data of a pixel corresponding to a count value witha bit indicating that count value. In other words, the counted-valueembedment processing section 260 embeds the count value into the pixeldata.

The counted-value embedment processing section 260 causes the framebuffer 270 to hold the output frame in which the count value isembedded.

The counted-value extraction section 280 extracts the embedded countvalue. The counted-value extraction section 280 reads the output framefrom the frame buffer 270 and also receives the invalid bit number fromthe camera control section 160. The counted-value extraction section 280extracts the count value per pixel from the output frame, based on theinvalid bit number. The counted-value extraction section 280 suppliesthe extracted count value to the stationary frame number count section220.

Further, the counted-value extraction section 280 replaces all the bitsindicating the count values with invalid bits and supplies the outputframe, which is subjected to the replacement, to the motiondetermination section 210 and the mixing section 250.

[Data Configuration Example of Pixel Data]

FIG. 12 is a diagram showing a data configuration example of pixel databefore the embedment of the count value in the second embodiment. Thedata size of the pixel data is 16 bits, for example. In the figure,white circles are valid bits, and crosses are bits indicating countvalues.

When a gain is smaller than 2×, all bits are valid in the pixel data.When a gain is 2× or larger and is smaller than 4×, one bit is aninvalid bit in the pixel data. When a gain is 4× or larger and issmaller than 8×, two bits are invalid bits in the pixel data. In such amanner, as the gain becomes larger, the number of invalid bitsincreases.

FIG. 13 is a diagram showing a data configuration example of pixel dataafter the embedment of the count value in the second embodiment. In thefigure, white circles are valid bits, and black circles are bitsindicating count values.

When a gain is smaller than 2×, since all bits are valid in the pixeldata, the count values cannot be embedded. When a gain is 2× or largerand is smaller than 4×, one bit is an invalid bit in the pixel data.Therefore, the count value is embedded in the one bit. When a gain is 4×or larger and is smaller than 8×, two bits are invalid bits in R datawithin the pixel data. Therefore, the count values are embedded in thetwo bits. In such a manner, as the gain becomes larger, the bit numberof the embedded count values can be increased.

By the embedment of the count value in the pixel data in such a manner,it is unnecessary to prepare a memory for holding the count value perpixel. It should be noted that the image processing section 200 may beconfigured to further include a memory for holding a count value perpixel and hold the count value in the memory. In this configuration, itis unnecessary to perform processing of embedding the count value andprocessing of extracting the count value.

[Configuration Example of Motion Determination Section]

FIG. 14 is a block diagram showing a configuration example of the motiondetermination section 210 in the second embodiment. The motiondetermination section 210 includes an entire-screen motion determinationsection 211 and a per-pixel motion determination section 213.

The entire-screen motion determination section 211 uses an inter-framedifference method, a block matching algorithm, or the like, to determinethe presence or absence of a motion on the entire screen. Theentire-screen motion determination section 211 supplies the motionamount dS to the mixing ratio supply section 230 and supplies adetermination result Mf on the presence or absence of a motion to thestationary frame number count section 220.

The per-pixel motion determination section 213 determines the presenceor absence of a motion per pixel. The per-pixel motion determinationsection calculates as a motion amount of pixel an absolute value of adifference between pixel values in corresponding pixels in the inputframe and the output frame. When such a motion amount is larger than apredetermined threshold value Th p, the per-pixel motion determinationsection 213 determines that there is a motion in that pixel. If not, theper-pixel motion determination section 213 determines that there is nomotion. The per-pixel motion determination section 213 suppliesdetermination results Mp1 to Mpm on the presence or absence of a motionper pixel to the stationary frame number count section 220.

It should be noted that the motion determination section 210 suppliesthe motion amount of the entire screen to the mixing ratio supplysection 230, but the motion determination section 210 is not limited tothis configuration. The motion determination section 210 can alsosupply, as the motion amount dS, statistics of the motion amount of theentire screen and the motion amount per pixel (for example, mean value)to the mixing ratio supply section 230.

[Configuration Example of Stationary Frame Number Count Section]

FIG. 15 is a block diagram showing a configuration example of thestationary frame number count section 220 in the second embodiment. Thestationary frame number count section 220 includes an entire-screenstationary frame number count section 221 and a per-pixel stationaryframe number count section 223.

The entire-screen stationary frame number count section 221 counts thenumber of times no motion is sequentially determined on the entirescreen. Specifically, the entire-screen stationary frame number countsection 221 holds the count value CNTf, and when it is determined thatthere is a motion in the determination result Mf from the entire-screenmotion determination section 211, sets the count value CNTf to be aninitial value. On the other hand, when it is determined that there is nomotion, the entire-screen stationary frame number count section 221increments the count value CNTf. The entire-screen stationary framenumber count section 221 then supplies the count value CNTf to themixing ratio supply section 230.

The per-pixel stationary frame number count section 223 counts, perpixel, the number of times no motion is sequentially determined.Specifically, the per-pixel stationary frame number count section 223receives the determination results Mp1 to Mpm from the per-pixel motiondetermination section 213 and receives the count values CNTp1 to CNTpmfrom the counted-value extraction section 280. When it is determinedthat there is a motion in each of the determination results, theper-pixel stationary frame number count section 223 sets a correspondingcount value to be an initial value. On the other hand, when it isdetermined that there is no motion, the per-pixel stationary framenumber count section 223 increments the corresponding count value. Theper-pixel stationary frame number count section 223 then supplies thecount values CNTp1 to CNTpm to the mixing ratio supply section 230.

[Configuration Example of Mixing Ratio Supply Section]

FIG. 16 is a block diagram showing a configuration example of the mixingratio supply section 230 in the second embodiment. The mixing ratiosupply section 230 of the second embodiment is different from the firstembodiment in that the mixing ratio supply section 230 further includesa minimum value selection section 241. The minimum value selectionsection 241 receives the count value CNTf and the count values CNTp1 toCNTpm from the stationary frame number count section 220. The minimumvalue selection section 241 supplies a minimum value MIN of those countvalues to the upper limit value generation section 242.

The upper limit value generation section 242 of the second embodimentcalculates the upper limit value L by the following expression, usingthe minimum value MIN instead of the CNTf.

L=(MIN+1)/(MIN+2)  Expression 16

It should be noted that the upper limit value generation section 242 cancalculate the upper limit value by using the following expressioninstead of Expression 16.

L=(MIN−1)/(MIN)  Expression 17

As described above, according to the second embodiment of the presenttechnology, the imaging apparatus 100 detects both of a motion of theentire screen and a motion per pixel, and thus can improve detectionaccuracy of a motion. By improvement in detection accuracy, the framesare mixed at an appropriate mixing ratio and a noise reducing effect isimproved.

3. Third Embodiment Configuration Example of Motion DeterminationSection

In the second embodiment, the image processing section 200 detects thepresence or absence of a motion of the entire screen as well as a motionper pixel, but a smaller unit of motion detection is desirable. An imageprocessing section 200 of a third embodiment is different from thesecond embodiment in that the image processing section 200 divides aframe into a plurality of areas and detects a motion per area instead ofa motion of the entire screen.

FIG. 17 is a block diagram showing a configuration example of the motiondetermination section 210 in the third embodiment. The motiondetermination section 210 of the third embodiment is different from thesecond embodiment in that the motion determination section 210 includesa per-area motion determination section 212 instead of the entire-screenmotion determination section 211.

The per-area motion determination section 212 divides each of the inputframe and the output frame into a plurality of areas and determines thepresence or absence of a motion per area. Here, it is assumed that eacharea has a predetermined shape (for example, square of 8×8 pixels) eachincluding a plurality of pixels.

The per-area motion determination section 212 uses a block matchingalgorithm, for example, to determine the presence or absence of a motionper area based on whether the motion amount dS is equal to or largerthan a threshold value Th_a. In the block matching, for example, eacharea is seen as a block, and a motion is detected per block. Theper-area motion determination section 212 supplies determination resultsMa1 to Mav of respective areas to the stationary frame number countsection 220. Here, v is an integer smaller than the number of pixels mand indicates the number of areas.

It should be noted that the size of the area and the size of the blockare not limited to the identical size. For example, the per-area motiondetermination section 212 may further divide each area into a pluralityof blocks and detect a motion based on a correlation of those blocks.

The configuration of the per-pixel motion determination section 213 ofthe third embodiment is similar to that of the second embodiment.

[Stationary Frame Number Count Section]

FIG. 18 is a block diagram showing a configuration example of thestationary frame number count section 220 in the third embodiment. Thestationary frame number count section 220 of the third embodiment isdifferent from the second embodiment in that the stationary frame numbercount section 220 includes a per-area stationary frame number countsection 222 instead of the entire-screen stationary frame number countsection 221.

The per-area stationary frame number count section 222 counts, per area,the number of times no motion is sequentially determined. The per-areastationary frame number count section 222 holds count values CNTa1 toCNTav in v areas. The per-area stationary frame number count section 222receives the determination results Ma1 to Mav from the per-area motiondetermination section 212. When it is determined that there is a motionin each of the determination results, the per-area stationary framenumber count section 222 sets a corresponding count value to be aninitial value. On the other hand, when it is determined that there is nomotion, the per-area stationary frame number count section 222increments the corresponding count value. The per-area stationary framenumber count section 222 then supplies the count values CNTa1 to CNTavto the mixing ratio supply section 230.

The configuration of the per-pixel stationary frame number count section223 of the third embodiment is similar to that of the second embodiment.

It should be noted that the imaging apparatus 100 detects both of amotion per area and a motion per pixel, but the imaging apparatus 100may be configured to detect any one of those motions.

Further, the imaging apparatus 100 detects both of a motion per area anda motion per pixel, but the imaging apparatus 100 may be configured todetect both of a motion of the entire screen and a motion per area.

As described above, according to the third embodiment of the presenttechnology, the imaging apparatus 100 detects both of a motion per areaand a motion per pixel, and thus can improve detection accuracy of amotion. By improvement in detection accuracy, the frames are mixed at anappropriate mixing ratio and a noise reducing effect is improved.

4. Fourth Embodiment Configuration Example of Motion DeterminationSection

In the second embodiment, the image processing section 200 detects bothof a motion of the entire screen and a motion per pixel, but it isdesirable to further detect a motion per area from a viewpoint ofdetection accuracy. An image processing section 200 of a fourthembodiment is different from the second embodiment in that the imageprocessing section 200 further detects a motion per area in addition toa motion of the entire screen and a motion per area.

FIG. 19 is a block diagram showing a configuration example of the motiondetermination section 210 in the fourth embodiment. The motiondetermination section 210 of the fourth embodiment is different from thesecond embodiment in that the motion determination section 210 furtherincludes a per-area motion determination section 212.

The configuration of the per-area motion determination section 212 ofthe fourth embodiment is similar to that of the third embodiment.

[Stationary Frame Number Count Section]

FIG. 20 is a block diagram showing a configuration example of thestationary frame number count section 220 in the fourth embodiment. Thestationary frame number count section 220 of the fourth embodiment isdifferent from the second embodiment in that the stationary frame numbercount section 220 further includes a per-area stationary frame numbercount section 222.

The configuration of the per-area stationary frame number count section222 of the fourth embodiment is similar to that of the third embodiment.

As described above, according to the fourth embodiment of the presenttechnology, the imaging apparatus 100 detects all of a motion of theentire screen, a motion per area, and a motion per pixel, and thus canimprove detection accuracy of a motion. By improvement in detectionaccuracy, the frames are mixed at an appropriate mixing ratio and anoise reducing effect is improved.

It should be noted that the embodiments described above are examples forembodying the present technology, and matters in the embodiments andmatters specifying the invention in the Claims have respectivecorrespondence relationships. Similarly, the matters specifying theinvention in the Claims and matters in the embodiments of the presenttechnology, which are denoted by names identical to the mattersspecifying the invention, have respective correspondence relationships.It should be noted that the present technology is not limited to theembodiments and can be embodied by variously modifying the embodimentswithout departing from the gist of the present technology.

Further, the processing procedures described in the above embodimentsmay be understood as a method including a series of those procedures.Alternatively, the processing procedures described in the aboveembodiments may be understood as a program for causing a computer toexecute the series of procedures or as a recording medium storing thatprogram. As the recording medium, for example, a CD (Compact Disc), anMD (Mini Disc), a DVD (Digital Versatile Disc), a memory card, a Blu-ray(registered trademark) Disc, or the like may be used.

It should be noted that the present technology can have the followingconfigurations.

(1) An image processing apparatus, including:

a determination section that determines, each time an input image isinput, whether a change amount of a pixel value distribution in theinput image and an output image that is output is smaller than apredetermined threshold value;

a mixing ratio supply section that supplies a value that becomes largeras the number of times the change amount is sequentially determined tobe smaller than the predetermined threshold value becomes larger, thevalue serving as a mixing ratio of the output image in mixing of theinput image and the output image; and

a mixing section that mixes, each time the input image is input, theinput image and the output image based on the supplied mixing ratio andoutputs the input image and the output image to serve as a new outputimage.

(2) The image processing apparatus according to (1), in which

the mixing ratio supply section includes

-   -   an upper limit value generation section that generates an upper        limit value that becomes larger as the number of times becomes        larger,    -   an acquisition section that acquires a value that becomes        smaller as the change amount becomes larger, and    -   a limitation section that supplies, when the acquired value is        larger than the upper limit value, the upper limit value to        serve as the mixing ratio, and supplies, when the acquired value        does not exceed the upper limit value, the acquired value to        serve as the mixing ratio.        (3) The image processing apparatus according to (1) or (2), in        which

the mixing ratio supply section supplies the mixing ratio that becomeslarger as the number of times becomes larger and at which a proportionof each of the input images mixed in the output image is made uniform.

(4) The image processing apparatus according to any one of (1) to (3),in which

each of the input image and the output image is an image including aplurality of areas each including a plurality of pixels,

the determination section determines, per area, whether the changeamount is smaller than the predetermined threshold value, and

the mixing ratio supply section acquires, per area, the number of timesthe change amount is sequentially determined to be smaller than thepredetermined threshold value, and supplies the mixing ratio thatbecomes larger as a minimum value of the numbers of times acquired perarea becomes larger.

(5) The image processing apparatus according to any one of (1) to (4),in which

each of the input image and the output image is an image including aplurality of pixels,

the determination section determines, per pixel, whether the changeamount is smaller than the predetermined threshold value, and

the mixing ratio supply section acquires, per pixel, the number of timesthe change amount is sequentially determined to be smaller than thepredetermined threshold value, and supplies the mixing ratio thatbecomes larger as a minimum value of the numbers of times acquired perpixel becomes larger.

(6) The image processing apparatus according to (5), further including:

a replacement section that replaces, when the pixel includes invaliddata together with valid data indicating a pixel value, the invalid datanot applying to the valid data, the invalid data within the pixel perpixel based on the number of times; and

an image holding section that holds the output image in which theinvalid data is replaced, in which

the mixing ratio supply section acquires the number of times from theheld output image.

(7) An imaging apparatus, including:

a determination section that determines, each time an input image isinput, whether a change amount of a pixel value distribution in theinput image and an output image that is output is smaller than apredetermined threshold value;

a mixing ratio supply section that supplies a value that becomes largeras the number of times the change amount is sequentially determined tobe smaller than the predetermined threshold value becomes larger, thevalue serving as a mixing ratio of the output image in mixing of theinput image and the output image;

a mixing section that mixes, each time the input image is input, theinput image and the output image based on the supplied mixing ratio andoutputs the input image and the output image to serve as a new outputimage; and

an image holding section that holds the output image that is output.

(8) An image processing method, including:

a determination procedure of determining, by a determination section,each time an input image is input, whether a change amount of a pixelvalue distribution in the input image and an output image that is outputis smaller than a predetermined threshold value;

a mixing ratio supply procedure of supplying, by a mixing ratio supplysection, a value that becomes larger as the number of times the changeamount is sequentially determined to be smaller than the predeterminedthreshold value becomes larger, the value serving as a mixing ratio ofthe output image in mixing of the input image and the output image; and

a mixing procedure of mixing, by a mixing section, each time the inputimage is input, the input image and the output image based on thesupplied mixing ratio and outputs the input image and the output imageto serve as a new output image.

(9) A program causing a computer to execute:

a determination procedure of determining, by a determination section,each time an input image is input, whether a change amount of a pixelvalue distribution in the input image and an output image that is outputis smaller than a predetermined threshold value;

a mixing ratio supply procedure of supplying, by a mixing ratio supplysection, a value that becomes larger as the number of times the changeamount is sequentially determined to be smaller than the predeterminedthreshold value becomes larger, the value serving as a mixing ratio ofthe output image in mixing of the input image and the output image; and

a mixing procedure of mixing, by a mixing section, each time the inputimage is input, the input image and the output image based on thesupplied mixing ratio and outputs the input image and the output imageto serve as a new output image.

DESCRIPTION OF REFERENCE NUMERALS

-   100 imaging apparatus-   110 imaging lens-   120 imaging device-   130 analog front end-   140 A/D conversion section-   150 programmable gain amplifier-   160 camera control section-   170 D/A conversion section-   180 display section-   200 image processing section-   210 motion determination section-   211 entire-screen motion determination section-   212 per-area motion determination section-   213 per-pixel motion determination section-   220 stationary frame number count section-   221 entire-screen stationary frame number count section-   222 per-area stationary frame number count section-   223 per-pixel stationary frame number count section-   230 mixing ratio supply section-   231 mixing ratio generation section-   241 minimum value selection section-   242 upper limit value generation section-   243 mixing ratio limitation section-   250 mixing section-   251 subtractor-   252, 253 multiplier-   254 adder-   260 counted-value embedment processing section-   270 frame buffer-   280 counted-value extraction section

1. An image processing apparatus, comprising: a determination sectionthat determines, each time an input image is input, whether a changeamount of a pixel value distribution in the input image and an outputimage that is output is smaller than a predetermined threshold value; amixing ratio supply section that supplies a value that becomes larger asthe number of times the change amount is sequentially determined to besmaller than the predetermined threshold value becomes larger, the valueserving as a mixing ratio of the output image in mixing of the inputimage and the output image; and a mixing section that mixes, each timethe input image is input, the input image and the output image based onthe supplied mixing ratio and outputs the input image and the outputimage to serve as a new output image.
 2. The image processing apparatusaccording to claim 1, wherein the mixing ratio supply section includesan upper limit value generation section that generates an upper limitvalue that becomes larger as the number of times becomes larger, anacquisition section that acquires a value that becomes smaller as thechange amount becomes larger, and a limitation section that supplies,when the acquired value is larger than the upper limit value, the upperlimit value to serve as the mixing ratio, and supplies, when theacquired value does not exceed the upper limit value, the acquired valueto serve as the mixing ratio.
 3. The image processing apparatusaccording to claim 1, wherein the mixing ratio supply section suppliesthe mixing ratio that becomes larger as the number of times becomeslarger and at which a proportion of each of the input images mixed inthe output image is made uniform.
 4. The image processing apparatusaccording to claim 1, wherein each of the input image and the outputimage is an image including a plurality of areas each including aplurality of pixels, the determination section determines, per area,whether the change amount is smaller than the predetermined thresholdvalue, and the mixing ratio supply section acquires, per area, thenumber of times the change amount is sequentially determined to besmaller than the predetermined threshold value, and supplies the mixingratio that becomes larger as a minimum value of the numbers of timesacquired per area becomes larger.
 5. The image processing apparatusaccording to claim 1, wherein each of the input image and the outputimage is an image including a plurality of pixels, the determinationsection determines, per pixel, whether the change amount is smaller thanthe predetermined threshold value, and the mixing ratio supply sectionacquires, per pixel, the number of times the change amount issequentially determined to be smaller than the predetermined thresholdvalue, and supplies the mixing ratio that becomes larger as a minimumvalue of the numbers of times acquired per pixel becomes larger.
 6. Theimage processing apparatus according to claim 5, further comprising: areplacement section that replaces, when the pixel includes invalid datatogether with valid data indicating a pixel value, the invalid data notapplying to the valid data, the invalid data within the pixel per pixelbased on the number of times; and an image holding section that holdsthe output image in which the invalid data is replaced, wherein themixing ratio supply section acquires the number of times from the heldoutput image.
 7. An imaging apparatus, comprising: a determinationsection that determines, each time an input image is input, whether achange amount of a pixel value distribution in the input image and anoutput image that is output is smaller than a predetermined thresholdvalue; a mixing ratio supply section that supplies a value that becomeslarger as the number of times the change amount is sequentiallydetermined to be smaller than the predetermined threshold value becomeslarger, the value serving as a mixing ratio of the output image inmixing of the input image and the output image; a mixing section thatmixes, each time the input image is input, the input image and theoutput image based on the supplied mixing ratio and outputs the inputimage and the output image to serve as a new output image; and an imageholding section that holds the output image that is output.
 8. An imageprocessing method, comprising: a determination procedure of determining,by a determination section, each time an input image is input, whether achange amount of a pixel value distribution in the input image and anoutput image that is output is smaller than a predetermined thresholdvalue; a mixing ratio supply procedure of supplying, by a mixing ratiosupply section, a value that becomes larger as the number of times thechange amount is sequentially determined to be smaller than thepredetermined threshold value becomes larger, the value serving as amixing ratio of the output image in mixing of the input image and theoutput image; and a mixing procedure of mixing, by a mixing section,each time the input image is input, the input image and the output imagebased on the supplied mixing ratio and outputs the input image and theoutput image to serve as a new output image.
 9. A program causing acomputer to execute: a determination procedure of determining, by adetermination section, each time an input image is input, whether achange amount of a pixel value distribution in the input image and anoutput image that is output is smaller than a predetermined thresholdvalue; a mixing ratio supply procedure of supplying, by a mixing ratiosupply section, a value that becomes larger as the number of times thechange amount is sequentially determined to be smaller than thepredetermined threshold value becomes larger, the value serving as amixing ratio of the output image in mixing of the input image and theoutput image; and a mixing procedure of mixing, by a mixing section,each time the input image is input, the input image and the output imagebased on the supplied mixing ratio and outputs the input image and theoutput image to serve as a new output image.